{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Basic Convolutional Neural Networks (CNN)\n",
    "- Objective: create basic CNN models with Keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://adeshpande3.github.io/assets/Cover.png\" style=\"width: 800px\"/>\n",
    "\n",
    "<br>\n",
    "- Fundamental CNN structures: CNNs are similar to MLPs since they only feed signals forward (feedforward nets), but have different kind of layers unique to CNNs\n",
    "    - ** Convolutional layer** : process data in a small receptive field (i.e., filter)\n",
    "    - ** Pooling layer** : downsample along 2 dimensions (usually width and height) \n",
    "    - ** Dense (fully connected) layer** : similar to hidden layers of MLPs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"http://cs231n.github.io/assets/cnn/convnet.jpeg\" style=\"width: 600px\"/>\n",
    "<br>\n",
    "<center> **ConvNet architecture** </center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn import datasets\n",
    "from sklearn.model_selection import train_test_split\n",
    "from keras.utils.np_utils import to_categorical"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Datset\n",
    "- Digits dataset in sklearn\n",
    "- Doc: http://scikit-learn.org/stable/auto_examples/datasets/plot_digits_last_image.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data = datasets.load_digits()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPgAAAD8CAYAAABaQGkdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAACxNJREFUeJzt3fuLXPUZx/HPp5vErRqTYqxKNjShaEAqNZqmhIjQBEus\nokJL3YCWSmGhoCiGihZL239A0h+KIFErmBpsVBDrBVsVK6QxF1M1txKDJRvURLwHTLLm6Q87gShp\n92zmnO+ZeXy/YHEvw36fQd45Z2ZnztcRIQA5fa3tAQA0h8CBxAgcSIzAgcQIHEiMwIHECBxIjMCB\nxAgcSGxKE790mk+JQZ3WxK9u1dissvfpnHPeL7bWvoMzi601OHqk2FpxZKzYWiV9poM6HIc80e0a\nCXxQp+n7XtbEr27Vez9eXHS9X61cW2yt32y+ptha59/2drG1xt55t9haJW2Iv1e6HafoQGIEDiRG\n4EBiBA4kRuBAYgQOJEbgQGIEDiRWKXDby23vsr3b9h1NDwWgHhMGbntA0h8lXSHpAkkrbF/Q9GAA\nulflCL5I0u6I2BMRhyWtlVTudY0ATlqVwGdL2nvc16Od7wHocbW92cT2iKQRSRrUqXX9WgBdqHIE\n3ydpznFfD3W+9wURcW9ELIyIhVN1Sl3zAehClcA3SjrP9jzb0yQNS3qi2bEA1GHCU/SIGLN9k6Rn\nJQ1Iuj8itjU+GYCuVXoMHhFPSXqq4VkA1IxXsgGJETiQGIEDiRE4kBiBA4kROJAYgQOJETiQWCM7\nm2RVcqcRSRqe/kGxtVbN/LTYWn/d8myxtS753S+LrSVJs+5dX3S9iXAEBxIjcCAxAgcSI3AgMQIH\nEiNwIDECBxIjcCAxAgcSq7Kzyf2299t+o8RAAOpT5Qj+J0nLG54DQAMmDDwiXpL0foFZANSMx+BA\nYmxdBCRW2xGcrYuA3sMpOpBYlT+TPSxpvaT5tkdt/6L5sQDUocreZCtKDAKgfpyiA4kROJAYgQOJ\nETiQGIEDiRE4kBiBA4kROJBY329dNLb0kmJrDU/fWmwtSbpi+XCxtWa8trPYWj99eVmxtd5f8Hmx\ntSRpVtHVJsYRHEiMwIHECBxIjMCBxAgcSIzAgcQIHEiMwIHECBxIjMCBxKpcdHGO7Rdsb7e9zfYt\nJQYD0L0qr0Ufk7QyIrbYni5ps+3nImJ7w7MB6FKVvcnejogtnc8/kbRD0uymBwPQvUm9m8z2XEkL\nJG04wc/YugjoMZWfZLN9uqRHJd0aER9/+edsXQT0nkqB256q8bjXRMRjzY4EoC5VnkW3pPsk7YiI\nu5sfCUBdqhzBl0i6QdJS21s7Hz9qeC4ANaiyN9nLklxgFgA145VsQGIEDiRG4EBiBA4kRuBAYgQO\nJEbgQGIEDiTW93uTfXZmubtw1/4Li60lSUcL7hdW0sbXv932CF8ZHMGBxAgcSIzAgcQIHEiMwIHE\nCBxIjMCBxAgcSIzAgcSqXHRx0PYrtv/V2bro9yUGA9C9Kq/zPCRpaUR82rl88su2n46IfzY8G4Au\nVbnoYkj6tPPl1M5HNDkUgHpU3fhgwPZWSfslPRcRJ9y6yPYm25uO6FDdcwI4CZUCj4jPI+IiSUOS\nFtn+zgluw9ZFQI+Z1LPoEfGhpBckLW9mHAB1qvIs+lm2Z3Y+/7qkyyXlfKMykEyVZ9HPlfSg7QGN\n/4PwSEQ82exYAOpQ5Vn01zS+JziAPsMr2YDECBxIjMCBxAgcSIzAgcQIHEiMwIHECBxIrP+3LvpG\nuX+j1qxfXGwtSTpfrxRdr5QpMw4XW2vso2nF1upFHMGBxAgcSIzAgcQIHEiMwIHECBxIjMCBxAgc\nSIzAgcQqB965NvqrtrkeG9AnJnMEv0XSjqYGAVC/qjubDEm6UtLqZscBUKeqR/BVkm6XdLTBWQDU\nrMrGB1dJ2h8Rmye4HXuTAT2myhF8iaSrbb8laa2kpbYf+vKN2JsM6D0TBh4Rd0bEUETMlTQs6fmI\nuL7xyQB0jb+DA4lN6oouEfGipBcbmQRA7TiCA4kROJAYgQOJETiQGIEDiRE4kBiBA4kROJBY329d\nNPhBuTe4fe/CN4utJUkfFVxryjlnF1vrugv+7/uWavXI05cWW6sXcQQHEiNwIDECBxIjcCAxAgcS\nI3AgMQIHEiNwIDECBxKr9Eq2zhVVP5H0uaSxiFjY5FAA6jGZl6r+ICLea2wSALXjFB1IrGrgIelv\ntjfbHmlyIAD1qXqKfmlE7LP9TUnP2d4ZES8df4NO+COSNKhTax4TwMmodASPiH2d/+6X9LikRSe4\nDVsXAT2myuaDp9mefuxzST+U9EbTgwHoXpVT9LMlPW772O3/HBHPNDoVgFpMGHhE7JH03QKzAKgZ\nfyYDEiNwIDECBxIjcCAxAgcSI3AgMQIHEiNwILG+37rojF3lNvj57dCTxdaSpJ+N3FZsranXHii2\nVknz7lzf9git4ggOJEbgQGIEDiRG4EBiBA4kRuBAYgQOJEbgQGIEDiRWKXDbM22vs73T9g7bi5se\nDED3qr5U9Q+SnomIn9ieJnHhc6AfTBi47RmSLpP0c0mKiMOSDjc7FoA6VDlFnyfpgKQHbL9qe3Xn\n+ugAelyVwKdIuljSPRGxQNJBSXd8+Ua2R2xvsr3piA7VPCaAk1El8FFJoxGxofP1Oo0H/wVsXQT0\nngkDj4h3JO21Pb/zrWWStjc6FYBaVH0W/WZJazrPoO+RdGNzIwGoS6XAI2KrpIUNzwKgZrySDUiM\nwIHECBxIjMCBxAgcSIzAgcQIHEiMwIHECBxIrO/3Jjv62s5ia113z8pia0nSXSsfLrbWqjeXFVtr\n40UDxdb6quMIDiRG4EBiBA4kRuBAYgQOJEbgQGIEDiRG4EBiBA4kNmHgtufb3nrcx8e2by0xHIDu\nTPhS1YjYJekiSbI9IGmfpMcbngtADSZ7ir5M0psR8Z8mhgFQr8m+2WRY0gnfAWF7RNKIJA2y+SjQ\nEyofwTubHlwt6S8n+jlbFwG9ZzKn6FdI2hIR7zY1DIB6TSbwFfofp+cAelOlwDv7gV8u6bFmxwFQ\np6p7kx2UdGbDswCoGa9kAxIjcCAxAgcSI3AgMQIHEiNwIDECBxIjcCAxR0T9v9Q+IGmybymdJem9\n2ofpDVnvG/erPd+KiLMmulEjgZ8M25siYmHbczQh633jfvU+TtGBxAgcSKyXAr+37QEalPW+cb96\nXM88BgdQv146ggOoWU8Ebnu57V22d9u+o+156mB7ju0XbG+3vc32LW3PVCfbA7Zftf1k27PUyfZM\n2+ts77S9w/bitmfqRuun6J1rrf9b41eMGZW0UdKKiNje6mBdsn2upHMjYovt6ZI2S7q23+/XMbZv\nk7RQ0hkRcVXb89TF9oOS/hERqzsXGj01Ij5se66T1QtH8EWSdkfEnog4LGmtpGtanqlrEfF2RGzp\nfP6JpB2SZrc7VT1sD0m6UtLqtmepk+0Zki6TdJ8kRcThfo5b6o3AZ0vae9zXo0oSwjG250paIGlD\nu5PUZpWk2yUdbXuQms2TdEDSA52HH6s71yPsW70QeGq2T5f0qKRbI+Ljtufplu2rJO2PiM1tz9KA\nKZIulnRPRCyQdFBSXz8n1AuB75M057ivhzrf63u2p2o87jURkeWKtEskXW37LY0/nFpq+6F2R6rN\nqKTRiDh2prVO48H3rV4IfKOk82zP6zypMSzpiZZn6ppta/yx3I6IuLvteeoSEXdGxFBEzNX4/6vn\nI+L6lseqRUS8I2mv7fmdby2T1NdPik52b7LaRcSY7ZskPStpQNL9EbGt5bHqsETSDZJet721871f\nR8RTLc6Eid0saU3nYLNH0o0tz9OV1v9MBqA5vXCKDqAhBA4kRuBAYgQOJEbgQGIEDiRG4EBiBA4k\n9l+8Q5/pEyhkXAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x22840818b70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "label:  0\n"
     ]
    }
   ],
   "source": [
    "plt.imshow(data.images[0])    # show first number in the dataset\n",
    "plt.show()\n",
    "print('label: ', data.target[0])    # label = '0'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_data = data.images\n",
    "y_data = data.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1797, 8, 8)\n",
      "(1797,)\n"
     ]
    }
   ],
   "source": [
    "# shape of data\n",
    "print(X_data.shape)    # (8 X 8) format\n",
    "print(y_data.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# reshape X_data into 3-D format\n",
    "# note that this follows image format of Tensorflow backend\n",
    "X_data = X_data.reshape((X_data.shape[0], X_data.shape[1], X_data.shape[2], 1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# one-hot encoding of y_data\n",
    "y_data = to_categorical(y_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# partition data into train/test sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X_data, y_data, test_size = 0.3, random_state = 777)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1257, 8, 8, 1)\n",
      "(540, 8, 8, 1)\n",
      "(1257, 10)\n",
      "(540, 10)\n"
     ]
    }
   ],
   "source": [
    "print(X_train.shape)\n",
    "print(X_test.shape)\n",
    "print(y_train.shape)\n",
    "print(y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras.models import Sequential\n",
    "from keras import optimizers\n",
    "from keras.layers import Dense, Activation, Flatten, Conv2D, MaxPooling2D"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 1. Creating model\n",
    "- Creating a model is same with MLP (Sequential)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model = Sequential()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Convolutional Layer\n",
    "- In general, 2D convolutional layer is used for image processing\n",
    "    - Size of filter (designated by 'kernel_size' parameter) defines **width and height of receptive field **\n",
    "    - Number of filters (designated by 'filters' parameter) is equal to **depth of the next layer**\n",
    "    - Strides (designated by 'strides' parameter) is** how far a filter makes change in its position** for each move\n",
    "    - Image can be **zero-padded** in order to prevent getting too small (designated by 'padding' parameter)\n",
    "- Doc: https://keras.io/layers/convolutional/\n",
    "<img src=\"http://cs231n.github.io/assets/cnn/depthcol.jpeg\" style=\"width: 400px\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# convolution layer\n",
    "model.add(Conv2D(input_shape = (X_data.shape[1], X_data.shape[2], X_data.shape[3]), filters = 10, kernel_size = (3,3), strides = (1,1), padding = 'valid'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Activation Layer\n",
    "- Identical to the activation layers in MLP\n",
    "- In general, relu is used as well\n",
    "- Doc: http://cs231n.github.io/assets/cnn/depthcol.jpeg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.add(Activation('relu'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Pooling layer\n",
    "- In general, max pooling method is used\n",
    "- Reduces number of parameters\n",
    "- Doc: https://keras.io/layers/pooling/\n",
    "<img src=\"http://cs231n.github.io/assets/cnn/maxpool.jpeg\" style=\"width: 600px\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.add(MaxPooling2D(pool_size = (2,2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Dense (fully connected layer)\n",
    "- Convolutional & pooling layers can be connected to dense layers\n",
    "- Sometimes, dense layers can be omitted\n",
    "- Doc: https://keras.io/layers/core/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# prior layer should be flattend to be connected to dense layers\n",
    "model.add(Flatten())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# dense layer with 50 neurons\n",
    "model.add(Dense(50, activation = 'relu'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# final layer with 10 neurons to classify the instances\n",
    "model.add(Dense(10, activation = 'softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Model compile & train\n",
    "- Identical to compiling MLP models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "adam = optimizers.Adam(lr = 0.001)\n",
    "model.compile(loss = 'categorical_crossentropy', optimizer = adam, metrics = ['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "history = model.fit(X_train, y_train, batch_size = 50, validation_split = 0.2, epochs = 100, verbose = 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt03HWd//HneyaTmUwubS5taZuWBGihUC4tpaCFAoJQ\nEEWQ6+qucFQW1AX37FlB96zoWf2tuyrL8luEH7rorgocfiiC/lAQLAKrsNCVS6HQFlpoek3TNkln\nMvfP74/PJJmkudlOms7k9TgnJ5nv95v5fj65vOYz7+/3+/macw4RESkvgYlugIiIFJ/CXUSkDCnc\nRUTKkMJdRKQMKdxFRMqQwl1EpAwp3EVEypDCXUSkDCncRUTKUMVE7bipqcm1tLRM1O5FRErSqlWr\ndjrnpo223YSFe0tLCy+99NJE7V5EpCSZ2btj2U5lGRGRMqRwFxEpQwp3EZEyNGE196Gk02na2tpI\nJBIT3ZSyEYlEaG5uJhQKTXRTROQgOqTCva2tjdraWlpaWjCziW5OyXPO0dHRQVtbG62trRPdHBE5\niEYty5jZvWa2w8xWD7PezOwOM1tvZq+a2eL9bUwikaCxsVHBXiRmRmNjo94JiUxCY6m5/xBYMcL6\nC4B5+Y/rgLsOpEEK9uLSz1Nkchq1LOOce8bMWkbY5GLgP52/X9/zZjbVzGY657YWqY1yCMjlHNu6\nErzbESeZye6z3gHxZJbd8RSdPWkioSBHNFXT2lSNGWzYGWPDzhipTI6WpmqOaKomGq5g484Y7+yM\n0d51AO8uzKgNVzA1GmJqtJJQcN8XNAfEkhl2x9N0xlOkMrn935/IAVrS0sDy+aNeh3RAilFznw1s\nKnjcll+2T7ib2XX40T1z584twq6La8+ePdx333189rOf/ZO+78ILL+S+++5j6tSpw27zla98heXL\nl3PuueceaDP7OOeIp7IkC4LKOUcm58gWfOzoTnLTv/yub5uqygqWHdnIB46ZzqK59eyKpdiwM8aW\nPT1EK4PUV1dSFQqyZmsXf9y0h5ff28M7O/eSSI9vIO7vm4z9uQ2w3tDIRLr+zCPHPdxtLDfIzo/c\nf+mcWzjEul8C33TOPZd//BRws3NuxMtPlyxZ4gZfobpmzRoWLFgw5sYX28aNG7noootYvXrg4YVM\nJkNFxfgde3bO7VM+SWVydCfSpLK5vpAOmBGuCBCuCJDI5NgTTw0I9kIBMyoCRjBgbHv3bX7yVrYv\n0Dr2plj13m6yOUdFwMjkhv8bqI1UcNKcqcyfUUtrUzUtjdVEw8Eht41WBqmPVjI1GmJvIsPGjhhv\nt8fAQes0P4oPBQNs3BljY0eMWDJLS2OU1mnVzKiNEAjsX+I65+hOZtgTS7OnJzVsf6orK6iPhpgS\nDRGuGLoPIoc6M1vlnFsy2nbFSKzNwJyCx835ZSXnlltu4e233+akk04iFAoRiUSor6/nzTffZO3a\ntXz0ox9l06ZNJBIJbrrpJq677jqgfyqFvXv3csEFF3D66afz+9//ntmzZ/PII49QVVXFNddcw0UX\nXcRll11GS0sLn/zkJ/nFL35BIpnin+76AUcedTR793TwN5/9FNu2buX4xafw/LMreeCx39HU1EQw\nYOScY3e8P8xrwhVMq41QHQ7SG4tmPtADBS8WmV1h7v7zgS+anT1pnl3XzmttncycEqF1Wg2zp1bR\nk/KllVgyw1HTazhyWs1+hW64JkhjTZiTD2/YZ92Jc6Zy4pzh3+X8qcyMukiIukiIuUSL9rwipawY\n4f4o8HkzewA4FegsRr39a794nTe2dB1w4wodO6uOWz983LDrv/nNb7J69Wpefvllnn76aT70oQ+x\nevXqvtMI7733XhoaGujp6eGUU07hYx/7GI2NjQOeY926ddx///1873vf44orruCnP/0pn/jEJ/bZ\nV82Ueu5/7Hf857//H358z518547v8o/f+SZL3n8Gn//rv+WlZ1fy8AM/YsHMWqZNq+v7vmzOkcpk\nCQYCVFbs/zVoU6pCXHTCLC46YdZ+P4eIHLpGDXczux84C2gyszbgViAE4Jy7G3gMuBBYD8SBa8er\nsQfb0qVLB5wffscdd/Dwww8DsGnTJtatW7dPuLe2tnLSSScBcOJJi3jtzXW07YqzN5Fha2eCN7Z0\nkc7mOOmM83A4zjn9fbyw8tc0N0R5bdULPPzww7ROq+HISz/MDfX1+5RrggGjqvKQujxhZNm0L4pX\nVA5cnstCbGdx9hEMQXTfdwjDyiShZ09x9v2nyiRg43Ow7nHY8AxMmQPzz4ejzoXurbD2CXj7KQjX\nwrzzYf55MO0YwAAHO9bAuidg7eOw6+3+561qgKPO8d9T3+KfY+3jsPVlcEOU7oKVcPj7/fZzT4Mt\nf/TP++4fYNrRfr9HfAA61vnn2fgsNC+F878ODUf451j3G3j872DnW0P3teYwmPdB379oY77dT8CO\n14v8Qx3CtAW+D/POg0Sn78O630D3luLvq6IKTrsBzvgbCNf0L+/e7vu87nHY8Kz/vcw/3//MZy2C\nwPhOEDCWs2WuHmW9Az5XtBbljTTCPliqq6v7vn766ad58skn+cMf/kA0GuWss84acP54Mp0lnc0R\nDodJZbJs70qyK54hHk/SncyQc46AQV1VBcGAceRh9bRMr2VvW5hMJjMR3RtZ52aom7Xvkcf4LqiI\nQOUI5Y+urfk/6ifgnad9uLSe6f/ZwnX5db+Bnl3Fa+/8FXD+/4LGIyGdgOfvhGdv88F44T/D7JMh\nl4OXfwxPfg3iRXph2V+1M2H+BbB7Azz7HXjmW355ZKoP6UQnvPh934/BgmFoXQ4LLwXLB8Tud2H9\nk/Da/81vZNC8BN5/I1SE932ORBe8/Vt4/Ev9y+pb4NgPw/bX4bdf9x8A1dPh8GX+d3bnqT7I2t+C\ntb+GxqNg+d/2t6OXc/6F4Y1H4Y8/8ssCFTD3fT4EA+M4QMlloO0l+MOd8F//6pdV1sKRZ/u/h2If\nTd+5Dp67DV65H878InRv639hBaibDcd8CDrW+9/z7/4Jlv6l/7scRyU0BBx/tbW1dHd3D7mus7OT\n+vp6otEob775Js8//zwAiXSWTM6xvn0v8ViMRDrLm9u6MTOqwxVEA2EWzKyjrirEjLoIzfVRAmbU\nVYX2qWUvW7aMBx98kJtvvpknnniC3bt396/MpPwr/YH+U6Ti/p9u5olDr9+1wY/G3vp/cOKfwUfu\n8CNjgJfvg0f/yrehdbkfFU093K9zWdj8P/4ffturflldM5xwBVjQj17W/sovjzb6721esm8o7I/u\nbfD8d+G7p8Hiv4D1T/nQPOqDsPUV+N45cOLV0L7Gj1DnnAZnfwmYgFNmLOBfaA47vj9k4rv8yLh6\nOjSfAsH87zgV86P7roLRZt1saD0DKqv3fe5c1vevcxO0nAHVTaO3Z/dG2PSi/3tomtffpu7t8O5z\nUN8KM0/yf3tdW+HJW31gVtbAB/8BTr1+33dlhbJpeO95SOzxfzORKWP6MRVFosv/XCtr/IvKSO08\nUKdeD7/6Ivzyr/3vuPkU+MDf+5H6jIUDf9frn4SGI8evLXkK9wKNjY0sW7aMhQsXUlVVxYwZM/rW\nrVixgrvvvpsFCxZw9NFHc9ppp7E7nmLdjr0455gTzdIZDFMRDDCjLkJDtJK6qhB7s0n/BLksJPf6\nUoTL+V9yunrAeXy33norV199NT/60Y943/vex2GHHUZtbS2ke2DnWgiEoGl+/z//SLJpSMf7Q8A5\nWPOoD+7OTbDsC3DOrf1vDVMxeO5f4L/ugEAQjr0YXrnPlwqu+E944W5Y+Q3/Dzr9WB/i654YuE8L\nwJxT/fPOOw9mHNf/R+2+5Ud76Vg+LIp8tsopn4Inv+pHu01Hw58/DEd+wP+DP/MteP4uH3aXfh+O\nv+zQOhcy2uB/3oNVVsPRF4z9eQJB/4LZPOqJFP3qW/zHYLUzYOHHBi6rmwmX3gNn3uxDeiwvHsGQ\nfzGaCJE6P2I+GOaeCp/5rX/H0DRv+DJhtMEPeA6CMZ0KOR4OxVMhR5PNObp60nQl0uxNZsjmHFMj\nFTS7zQTScT8inTKnPziyKR/miU5fax1KsNKXKoIhkskkwWCQipoG/vDSK9xwww28vOpFH+wu518g\nKqP+rXDviDfZ7deF6/r3m4r7emzOl3vWbO5iwep/9iOx6cf50H3tQVh4GXz0u/DmL+GJv4euzXD8\n5XDu12DKbPjjj+EXN/nn7tnlR78fvsOPgJyDXe8MrF03tP5pte/xsLcdqqb2v9vo1bPbl5NCVRPT\nLpEiOZinQk4KqUyWDTv91ZmhYIApkRBTqkLU9GzGEnE/kol3+BFz/eEQ64C923wIVlb7t9PhWl+i\nAB/IqW4/sozvAnK89857XHH9zeRyOSojVXzvrrt8gOYy0DjPv0DseRf2bPIjq87NkMyfUVRZDVOa\nIZvxJYlAhX9LnY6D2+NLMRd+G06+1o/wpi+Ap74G76z07T7sePjY9/1Btl6LPuFrww9fD8u/CGd/\nuf8FxMzXtw81NcNcGFJVf3DbITLBNHIfg55Uhg0dcWrdXmaE04SidVi4Dvbu8AFeOxNqD4NYO3S2\n0XdmQ7jOB+5QB7QKFf4Ocln/nLH2/mX1rX40Cr5M0r3Nf20Bv18L+uX5kToVVdB4hH9XwAg/11cf\n9AfzTv1LWPzJ4Uslzh1aZQyRSUwj9/2Uc45Nu+LEklnCFf5c8s6eNNNtD9PZBUmD5C76AryqAWry\ntfnqaT5Q97ZDzXRf8xuLwuAMVvgXhGijD+xwXX+wgz+9LJfzBzBrZ/aXH6qm+tDPZXxpaCw17ROu\nGFv9T8EuUnIU7gWcc2zr6GRKcgczAjliuSidiSrmBLqZ4jr9W/spc/wBzmSnD9kpsweGX2RKcc4I\nCFX1n09cyMzvc7BA/kVBRASFex+Xy7J352YOS+8CMwLBMJFMB43gpxSsmeFHymb+QoXCixVERA4x\nkzvcsylIdOESXbhkN7XkiFfUUdU0x5dXsml/wDJQcXDPzxUROUCT7wbZ6R5/UciON/2VeJ2byCRj\n7HbVbK+cS9X0I7D8gUh/WXvjsMFeU+NH71u2bOGyyy4bcpuzzjqLwQeOB7v99tuJx+N9jy+88EL2\n7Jmgy+NFpCxMrnBP7vUX0uzdDoEAsfB01rlm1jKXQP1cpjc27Nedi2bNmsVDDz20380aHO6PPfbY\niHPDi4iMZvKEeybhzxmvqIQZC+muaeXtnmoqwlHmz6ilPlrJl770Je68s38uj69+9at8/etf55xz\nzmHx4sUcf/zxPPLII/s89caNG1m40E9139PTw1VXXcWCBQu45JJL6Onp6dvuhhtuYMmSJRx33HHc\neuutgJ+MbMuWLZx99tmcffbZgJ9CeOdOP/fJbbfdxsKFC1m4cCG333573/4WLFjAZz7zGY477jjO\nO++8AfsRETl0a+6/ugW2vVakJ3P+Yp7Go+DiO0kTZNOuvURCQQ5viPbN8XLllVfyhS98gc99zs+D\n9uCDD/L4449z4403UldXx86dOznttNP4yEc+MuwI/6677iIajbJmzRpeffVVFi/uv1/4N77xDRoa\nGshms5xzzjm8+uqr3Hjjjdx2222sXLmSpqaBl3OvWrWKH/zgB7zwwgs45zj11FM588wzqa+vH/PU\nwiIyOU2CkbvzdXbnLypywUo27+4h6xxzCoIdYNGiRezYsYMtW7bwyiuvUF9fz2GHHcaXv/xlTjjh\nBM4991w2b97M9u3bh93bM8880xeyJ5xwAieccELfugcffJDFixezaNEiXn/9dd54440RW/7cc89x\nySWXUF1dTU1NDZdeeinPPvssMHBq4ZNPPpmNGzfu7w9IRMrQoTtyv+CbxXme7m3+YqCpcyHaSMfe\nJF2JNLOmVlEV2vdCn8svv5yHHnqIbdu2ceWVV/KTn/yE9vZ2Vq1aRSgUoqWlZcBUv2O1YcMGvv3t\nb/Piiy9SX1/PNddcs1/P0ysc7r/qNRgMqiwjIgOU98g9HffhHpkK0UZyzrGtM0FtJERj9dDTf155\n5ZU88MADPPTQQ1x++eV0dnYyffp0QqEQK1eu5N133x1xl8uXL+e+++4DYPXq1bz6qp/+tquri+rq\naqZMmcL27dv51a9+1fc9w001fMYZZ/Dzn/+ceDxOLBbj4Ycf5owzJmiGPREpKYfuyP1AuRzsfs9f\nhj/F3+K1J5Ul5xwN1aFha+bHHXcc3d3dzJ49m5kzZ/Lxj3+cD3/4wxx//PEsWbKEY445ZsTd3nDD\nDVx77bUsWLCABQsWcPLJJwNw4oknsmjRIo455hjmzJnDsmXL+r7nuuuuY8WKFcyaNYuVK1f2LV+8\neDHXXHMNS5cuBeDTn/40ixYtUglGREZVvhOHdW3xpzw2HNF3nvqO7gTbOhMsmFlHKFjeb1oKHaoT\nsonIn26sE4eVZ8Jl0z7YqxoGXIAUT2YJVwQnVbCLyORUninXe2OMghtHOOeIpTJUh4t8ByARkUPQ\nIRfuRSkT9YZ7wTzqiXSObM5RXVm+hxmGMlFlNxGZWIdUuEciETo6Og48kDJJfyOLQP+t1mIpfyOL\nyTRyd87R0dFBJBKZ6KaIyEF2SA1jm5ubaWtro729ffSNR7J3hz9bZs+bfYt2xVIkMzne7p5cQReJ\nRGhu1jzvIpPNIRXuoVCI1tbWA3+i2y6DlmX+Tu34Eexp//gUS1sb+d9X66wRESl/h1RZpihSMehq\n8zeUztu0q4ftXUmWtugmySIyOZRfuHes95+b+sP9vzfuAuCU1oahvkNEpOyUX7jvXOc/N83vW/Ti\nhl1MqQoxf3rtBDVKROTgKtNwtwE3l35x4y6WHF4/YAZIEZFyVobhvhbqD4eQPytme1eCd3bGWKqS\njIhMImUY7usGlGSeWetPqzx9XtNw3yEiUnbKK9xzOX9AteBMmWfW7aSpJsyCw+omsGEiIgdXeYV7\nVxtkevrOlMnmHM+ta2f5vCbV20VkUimvcN+51n/Ol2VWb+5kdzzN8vnTJrBRIiIHX5mF+8Bz3FVv\nF5HJqszCfa2fv73aj9SfWdfOwtl1NNWER/lGEZHyUn7h3jQfzOhKpPmf9/awfJ5KMiIy+Ywp3M1s\nhZm9ZWbrzeyWIdZPMbNfmNkrZva6mV1b/KaOQcGZMr9f30E251RvF5FJadRwN7MgcCdwAXAscLWZ\nHTtos88BbzjnTgTOAr5jZpVFbuvIEl3QvbW/3r6unerKIIvnarIwEZl8xjJyXwqsd86945xLAQ8A\nFw/axgG1ZmZADbALyBS1paPZ867/3HAEzjmeWdvO+45sorKivCpPIiJjMZbkmw1sKnjcll9W6N+A\nBcAW4DXgJudcrigtHKuePf5zVT3buhK07e5h2VGNB7UJIiKHimINa88HXgZmAScB/2Zm+1wSambX\nmdlLZvbSAd9tabBkt/8cqWNrp7+H6uGN0eLuQ0SkRIwl3DcDcwoeN+eXFboW+Jnz1gMbgGMGP5Fz\n7h7n3BLn3JJp04p8oDPZ5T+H69jR5cN9eu3kuqWeiEivsYT7i8A8M2vNHyS9Cnh00DbvAecAmNkM\n4GjgnWI2dFS9I/dwHTu6kwDMqFO4i8jkNOo9VJ1zGTP7PPA4EATudc69bmbX59ffDfwD8EMzew0w\n4Gbn3M5xbPe+Ep3+c6SO7V0dBANGY/XBPWFHRORQMaYbZDvnHgMeG7Ts7oKvtwDnFbdpf6JkNwQr\noSLM9q4k02rCmixMRCat8jlPMNkFYX8Md3tXghl1mnJARCav8gn3RBeE/T1S27uTTFe9XUQmsfIJ\n92Q3RDRyFxGBsgp3X5ZJZrLsjqd1GqSITGplFO7d+XPce0+D1MhdRCav8gn3fM19R3f+AibV3EVk\nEiufcE92QaRg5K6yjIhMYuUR7s7lyzK1bO+dekBlGRGZxMoj3NNxcFkI17G9O0lFwGiI6upUEZm8\nyiPcE/lJwyJ1bO9KML1WV6eKyORWHuFeMGmYLmASESmbcO+f7lcXMImIlEu4984IGa5le1dSFzCJ\nyKRXHuGeL8skK2ro7Elr5C4ik16ZhLsvy3Sk/RkyqrmLyGRXJuHuR+7bk37ErjswichkVx7hnj8V\nckvC33tkeq3KMiIyuZVHuCe7obKG7d1pQCN3EZEyCffO/KRhSUJBoz4amugWiYhMqDIJ997pfhNM\nr41gpqtTRWRyK49wz0/3u71bFzCJiEC5hHv+Fnu6gElExCuTcO/qm+5XI3cRkbIJ924yoVq6Exld\nwCQiQrmEe6KLuFUDOg1SRATKIdyzGUjHiFkVAA3VOg1SRKT0wz3lpx6I5UfudRGFu4hI6Yd7fuqB\nGH7kXlelcBcRKf1wz08a1uWigEbuIiJQFuHuR+6dOX8gtTZSMZGtERE5JJRBuPuR++5shGDAiFYG\nJ7hBIiITr/TDPV9z35WJUBup0LwyIiJA6dcweu/ClAlTF9GoXUQEyijcd6QqqY1o1C4iAmUR7t1g\nQTqSQY3cRUTyyqPmHq6lK5HVmTIiInmlH+756X67E2ldwCQikjemcDezFWb2lpmtN7NbhtnmLDN7\n2cxeN7PfFbeZI0h2QbiOrkRGI3cRkbxR09DMgsCdwAeBNuBFM3vUOfdGwTZTge8CK5xz75nZ9PFq\n8D4SXbhwLXuTGV2dKiKSN5aR+1JgvXPuHedcCngAuHjQNn8G/Mw59x6Ac25HcZs5gmQXmVAtoHll\nRER6jSXcZwObCh635ZcVmg/Um9nTZrbKzP5iqCcys+vM7CUze6m9vX3/WjxYsotUhZ8RUmUZERGv\nWAdUK4CTgQ8B5wN/b2bzB2/knLvHObfEObdk2rRpxdlzsptksAbQpGEiIr3GMtTdDMwpeNycX1ao\nDehwzsWAmJk9A5wIrC1KK4fjHCS66LHeGSE1chcRgbGN3F8E5plZq5lVAlcBjw7a5hHgdDOrMLMo\ncCqwprhNHUImCbl03y32VHMXEfFGHeo65zJm9nngcSAI3Ouce93Mrs+vv9s5t8bMfg28CuSA7zvn\nVo9nw4G+qQf25m/UoZq7iIg3pjR0zj0GPDZo2d2DHn8L+FbxmjYGvTfqQDfqEBEpVNpXqCY6AejM\n+ht11GjkLiIClHq450fundkI0cogoWBpd0dEpFhKOw3TcQB2Z0Oqt4uIFCjtcE/FANidDqneLiJS\noLTDPT9y35UKauQuIlKgtMM95cN9Z7JC57iLiBQo7XDPj9zbk0GVZURECpRBuBs7EwGVZURECpR2\nuKfiuFCU7mRGZRkRkQKlHe7pGISipLNOI3cRkQKlHe6pOLmQn1dGNXcRkX6lHe7pOJmgJg0TERms\nbMJdNXcRkX6lHe6pOCnzk4bpRh0iIv1KO9zTMZKB3nDXyF1EpFeJh3sPifzIvVbhLiLSp7TDPRWn\nx4UBqKtSWUZEpFdph3s6RtxVUhEwqkLBiW6NiMgho7TDPRUn5iqpjVRgZhPdGhGRQ0bphns2Dbk0\n3bmwToMUERmkdMM9PyNkl+7CJCKyj9IN9/xc7t0Z3YVJRGSw0g333vunZjRyFxEZrHTDXfdPFREZ\nVumGe7oH6L1/qsJdRKRQCYe7H7nvSod0AZOIyCClG+75A6o9hDVyFxEZpHTDPd0f7poRUkRkoJIP\n97jTyF1EZLDSDfd8WSaukbuIyD5KN9zzB1QThKkOK9xFRAqVbrin4uQsSIoKopWaEVJEpFDphnvf\n/VONKoW7iMgApR3uAX9z7GilyjIiIoVKN9xTcVJBf4s9lWVERAYq3XBPx0lZGDMIV5RuN0RExsOY\nUtHMVpjZW2a23sxuGWG7U8wsY2aXFa+Jw0jFSFqEaCiouzCJiAwyaribWRC4E7gAOBa42syOHWa7\nfwKeKHYjh5TuIWERqlRvFxHZx1hG7kuB9c65d5xzKeAB4OIhtvsr4KfAjiK2b3jpOD2EVW8XERnC\nWMJ9NrCp4HFbflkfM5sNXALcVbymjSIVo8cp3EVEhlKsI5G3Azc753IjbWRm15nZS2b2Unt7+4Ht\nMR0nTljnuIuIDGEsBevNwJyCx835ZYWWAA/kD2w2AReaWcY59/PCjZxz9wD3ACxZssTtb6MBSPcQ\nC1Rq5C4iMoSxhPuLwDwza8WH+lXAnxVu4Jxr7f3azH4I/HJwsBeVc5CK0V1ZSVVIB1RFRAYbNRmd\ncxkz+zzwOBAE7nXOvW5m1+fX3z3ObdxXNgUuy96cRu4iIkMZ07DXOfcY8NigZUOGunPumgNv1ijy\nN8fuzircRUSGUpqXduZv1NGZDemAqojIEEo03HsA6MyENHIXERlCaYZ7viwTc2HNCCkiMoTSDPeC\nm2Nr5C4isq/SDPdU/82xFe4iIvsqzXAvGLlr4jARkX2VdLjHCRMNaeQuIjJYaYZ7/oCqJg4TERla\naYb7gLKMwl1EZLASDXd/nnscnQopIjKU0gz3VIxsIESWoMoyIiJDKM1wT8fJBKoAVJYRERlCaYZ7\nKk46EAHQyF1EZAilGe7pGKl8uEcqFO4iIoOVaLj3kAxEqAoFCQRsolsjInLIKc1wT8VIWkQlGRGR\nYZRmuKfjJHSOu4jIsEoz3FNxzQgpIjKC0gz3dJyY06RhIiLDKdlwjztNGiYiMpzSDPdUnFhOt9gT\nERlO6YW7c5CO053TAVURkeGUXrhnEoCjWyN3EZFhlV6452+x15Wt1IyQIiLDKL1wT/sbdXRmQirL\niIgMo/TCPT9yj+UqdbaMiMgwSi/c8yP3OGGiYZVlRESGUoLh3nsXJs0tIyIynNIL93xZRjfHFhEZ\nXumFe74s00MlVaq5i4gMqfTCfdZi3ln2Lba4Jp0KKSIyjNIL9/rDeW/uR+kmqlMhRUSGUXrhDsRT\nWUD3TxURGY7CXUSkDJVkuPekMgAqy4iIDKMkw71/5K4DqiIiQxlTuJvZCjN7y8zWm9ktQ6z/uJm9\namavmdnvzezE4je1X2+461RIEZGhjRruZhYE7gQuAI4FrjazYwdttgE40zl3PPAPwD3FbmihnnSW\ncEWAYMDGczciIiVrLCP3pcB659w7zrkU8ABwceEGzrnfO+d25x8+DzQXt5kDxVMZHUwVERnBWMJ9\nNrCp4HE9zXSGAAAFd0lEQVRbftlwPgX86kAaNZp4Kqt6u4jICIqakGZ2Nj7cTx9m/XXAdQBz587d\n7/30pLI6U0ZEZARjGblvBuYUPG7OLxvAzE4Avg9c7JzrGOqJnHP3OOeWOOeWTJs2bX/aC/SO3BXu\nIiLDGUu4vwjMM7NWM6sErgIeLdzAzOYCPwP+3Dm3tvjNHKgnldWZMiIiIxi1LOOcy5jZ54HHgSBw\nr3PudTO7Pr/+buArQCPwXTMDyDjnloxXo+PpDNNqwuP19CIiJW9MNXfn3GPAY4OW3V3w9aeBTxe3\nacPTAVURkZGV5BWqOqAqIjKykgx3HVAVERlZSYa7Ru4iIiMruXDPZHOksjmiIdXcRUSGU3LhHk/7\nScOqwxq5i4gMp+TCvad3RkiVZUREhlVy4a67MImIjK4Ewz1/FybV3EVEhlVy4d6jkbuIyKhKLtxV\nlhERGV3JhrsOqIqIDK/kwn1abSUXLDyMxmpNHCYiMpySOyp58uENnHx4w0Q3Q0TkkFZyI3cRERmd\nwl1EpAwp3EVEypDCXUSkDCncRUTKkMJdRKQMKdxFRMqQwl1EpAyZc25idmzWDry7n9/eBOwsYnNK\nxWTs92TsM0zOfk/GPsOf3u/DnXPTRttowsL9QJjZS865JRPdjoNtMvZ7MvYZJme/J2OfYfz6rbKM\niEgZUriLiJShUg33eya6ARNkMvZ7MvYZJme/J2OfYZz6XZI1dxERGVmpjtxFRGQEJRfuZrbCzN4y\ns/VmdstEt2c8mNkcM1tpZm+Y2etmdlN+eYOZ/cbM1uU/1090W4vNzIJm9kcz+2X+8WTo81Qze8jM\n3jSzNWb2vknS77/O/32vNrP7zSxSbv02s3vNbIeZrS5YNmwfzexL+Wx7y8zOP5B9l1S4m1kQuBO4\nADgWuNrMjp3YVo2LDPA3zrljgdOAz+X7eQvwlHNuHvBU/nG5uQlYU/B4MvT5X4FfO+eOAU7E97+s\n+21ms4EbgSXOuYVAELiK8uv3D4EVg5YN2cf8//hVwHH57/luPvP2S0mFO7AUWO+ce8c5lwIeAC6e\n4DYVnXNuq3Puf/Jfd+P/2Wfj+/of+c3+A/joxLRwfJhZM/Ah4PsFi8u9z1OA5cC/AzjnUs65PZR5\nv/MqgCozqwCiwBbKrN/OuWeAXYMWD9fHi4EHnHNJ59wGYD0+8/ZLqYX7bGBTweO2/LKyZWYtwCLg\nBWCGc25rftU2YMYENWu83A58EcgVLCv3PrcC7cAP8uWo75tZNWXeb+fcZuDbwHvAVqDTOfcEZd7v\nvOH6WNR8K7Vwn1TMrAb4KfAF51xX4TrnT3Mqm1OdzOwiYIdzbtVw25Rbn/MqgMXAXc65RUCMQaWI\ncux3vs58Mf7FbRZQbWafKNymHPs92Hj2sdTCfTMwp+Bxc35Z2TGzED7Yf+Kc+1l+8XYzm5lfPxPY\nMVHtGwfLgI+Y2UZ8ue0DZvZjyrvP4Ednbc65F/KPH8KHfbn3+1xgg3Ou3TmXBn4GvJ/y7zcM38ei\n5luphfuLwDwzazWzSvzBh0cnuE1FZ2aGr8Gucc7dVrDqUeCT+a8/CTxysNs2XpxzX3LONTvnWvC/\n19865z5BGfcZwDm3DdhkZkfnF50DvEGZ9xtfjjnNzKL5v/dz8MeWyr3fMHwfHwWuMrOwmbUC84D/\n3u+9OOdK6gO4EFgLvA383US3Z5z6eDr+rdqrwMv5jwuBRvzR9XXAk0DDRLd1nPp/FvDL/Ndl32fg\nJOCl/O/750D9JOn314A3gdXAj4BwufUbuB9/TCGNf5f2qZH6CPxdPtveAi44kH3rClURkTJUamUZ\nEREZA4W7iEgZUriLiJQhhbuISBlSuIuIlCGFu4hIGVK4i4iUIYW7iEgZ+v+JmEVL0942ZgAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x22840f25710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(history.history['acc'])\n",
    "plt.plot(history.history['val_acc'])\n",
    "plt.legend(['training', 'validation'], loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "540/540 [==============================] - 0s     \n"
     ]
    }
   ],
   "source": [
    "results = model.evaluate(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test accuracy:  0.972222222222\n"
     ]
    }
   ],
   "source": [
    "print('Test accuracy: ', results[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even simple CNN model shows fine performance of **97% **test accuracy"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
